Statistical cache promotion

ABSTRACT

Storing data in a cache is disclosed. It is determined that a data record is not stored in a cache. A random value is generated using a threshold value. It is determined whether to store the data record in the cache based at least in part on the generated random value.

BACKGROUND OF THE INVENTION

Caches allow data to be stored at a location that enables faster accessto the data. For example, the cache may be a faster storage and/orlocated closer to a desired location. Often caches are smaller in sizeto be cost effective and efficient in performance. This requires thedata stored in the cache to be a duplicate store of an original datastored elsewhere in a larger storage. If a requested data is stored in acache, the requested data can be provided using the data in the cache.However, if the requested data is not stored in the cache, the data hasto often be obtained from another slower cache and/or an originalrepository of the data, all of which are comparatively slower. Thus, itis desirable to serve data requests from the fastest cache as often aspossible by placing in the cache data that will be likely requested inthe future. One way of predicting which data might be requested from thecache is to place in the cache the data that has been utilized the mostin the past. Previously, determining the most utilized data requiredstoring a counter for each data that may be placed in the cache anditerating the counter every time an associated data is accessed.However, storing and maintaining a counter for each potentially accesseddata consumes valuable storage and computing resources. Therefore thereexists a need for a better way to determine which data to store in acache.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1 is a block diagram illustrating an embodiment of a system forcaching data.

FIG. 2 is a flow chart illustrating an embodiment of a process forprocessing a request for a data record.

FIG. 3 is a flow chart illustrating an embodiment of a process fordetermining whether to cache a data record in a cache storage.

FIG. 4 is a flow chart illustrating an embodiment of a process fordetermining a threshold value.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess; an apparatus; a system; a composition of matter; a computerprogram product embodied on a computer readable storage medium; and/or aprocessor, such as a processor configured to execute instructions storedon and/or provided by a memory coupled to the processor. In thisspecification, these implementations, or any other form that theinvention may take, may be referred to as techniques. In general, theorder of the steps of disclosed processes may be altered within thescope of the invention. Unless stated otherwise, a component such as aprocessor or a memory described as being configured to perform a taskmay be implemented as a general component that is temporarily configuredto perform the task at a given time or a specific component that ismanufactured to perform the task. As used herein, the term ‘processor’refers to one or more devices, circuits, and/or processing coresconfigured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

Determining whether to store a data record in a cache is disclosed. Insome embodiments, the stored data record may be any type of dataincluding a database record, a key-value pair, a file, a text, a value,an image, and any other type of data. In some embodiments, when it isdetermined that a requested data record is not stored in a cache, it isdetermined whether the requested data record should be placed in thecache to serve a future request for the data record. For the requesteddata record, a random value is generated using a determined and/orprovided threshold value. Based at least in part on the random value, itis determined whether to store the data record in the cache. In someembodiments, if the random value and/or a calculation performed usingthe random value matches a predetermined value, the data record isstored in the cache. For example, the probability that the random valueand/or a calculation performed using the random value matches thepredetermined value approximates a desired probability that the datarecord should be stored in the cache each time a determination is made.In some embodiments, a frequently accessed data record will have ahigher probability of being placed in the cache than an infrequentlyaccessed data record because each time a data record that is not storedin the cache is requested, another probabilistic determination is madeon whether to store the data record in the cache. Each time anadditional probabilistic determination is made on whether to store thedata record in the cache, the overall probability that the data recordwill be stored in the cache increases. In some embodiments, thethreshold value that is used to determine the random value controls thelikelihood the data record will be stored in the cache. The thresholdvalue may be determined dynamically and/or predetermined based on anobserved data access pattern. In some embodiments, the threshold valueis determined based at least in part on a size and/or hierarchy level ofa cache of interest.

FIG. 1 is a block diagram illustrating an embodiment of a system forcaching data. Requestor 102 requests a data record from cache manager104. Examples of requestor 102 include any hardware or software system,component, process, and/or application. Cache manager 104 managesstorage grouping 106. Cache manager 104 may include one or more hardwareor software systems, components, processes, and/or applications. In someembodiments, managing storage grouping 106 includes obtaining, reading,storing, writing, removing, deleting, and/or replacing data in storagegrouping 106. Storage grouping 106 includes L1 cache 108, L2 cache 112and storage 114. The components included in storage grouping 106 may beincluded in one or more physically separate systems. For example, L1cache 108 and L2 cache 112 may be included in different physicalsystems. An example of storage grouping 106 has been shown in FIG. 1 toillustrate an example of at least a portion of caches and storages thatmay contain a data requested by requested by requestor 102 and/orcomponents at least in part managed by cache manager 104. In the exampleshown, L1 cache 108 is the first cache level, L2 cache 112 is the secondcache level, and storage 114 is the backing repository that contains thepotential data that may be stored in the caches. For example, when adata is requested by requestor 102, cache manager 104 first checks L1cache 108 to determine whether the requested data is stored in L1 cache108. If the requested data is stored in L1 cache 108, the requested datais provided from L1 cache 108. If the requested data is not stored in L1cache 108, cache manager checks L2 cache 112 to determine whether therequested data is stored in the L2 cache. If the requested data is notstored in L2 cache 112, the requested data is obtained from storage 114and provided to the requestor. In some embodiments, whenever a requesteddata is determined to be not stored in cache, cache manager 104determines whether the requested data should be stored in the cache toserve a future request. In some embodiments, cache manager 104determines which requested data must be cached and which requested datais not required to be cached. For example, cache manager 104 isconfigured to cache all requested data of a predetermined type, and ifthe requested data is not of the predetermined type, cache manager 104makes a probabilistic determination on whether to cache the requestdata.

In various embodiments, other levels of caches may exist. For example,cache level(s) above L1 cache 108 and/or cache level(s) below L2 cache112 may exist. These additional cache levels may be managed by cachemanager 104 and/or another cache manager. In some embodiments, L1 cache108 is associated with a faster performance (e.g., access time, transferrate, write/read speed, etc.) as compared to L2 cache 112. In someembodiments, L2 cache 112 is associated with a faster performance ascompared to storage 114. In some embodiments, L1 cache 108 is smaller insize as compared to the size of L2 cache 112. In some embodiments, L2cache 112 is smaller in size as compared to the size of storage 114.Examples of storage 114 include a database, a production storage, abackup storage, a physical disk, a flash storage, a memory, a harddriver, a storage media, and/or any medium for storing data. Examples ofL1 cache 108 and/or L2 cache 112 include a memory, a flash storage, aprocessor cache, and any other medium for storing data.

In various embodiments, the components shown in FIG. 1 may exist invarious combinations of hardware machines. One or more of the componentsshown in FIG. 1 may be included in the same machine. For example, cachemanager 104 and storage group 106 may be included in the same server. Inthe example shown in FIG. 1, portions of the communication path betweenthe components are shown. Other communication paths may exist and theexample of FIG. 1 has been simplified to illustrate the example clearly.The components of FIG. 1 may communicate with another component via anetwork. Examples of the network include one or more of the following: adirect or indirect physical communication connection, mobilecommunication network, Internet, intranet, Local Area Network, Wide AreaNetwork, Storage Area Network, and any other form of connecting two ormore systems, components, or storage devices together. Although singleinstances of components have been shown to simplify the diagram,additional instances of any of the components shown in FIG. 1 may exist.For example, additional instances of L1 cache and/or L2 cache that storedata in storage 114 may exist. These additional caches may be managed bycache manager 104 and/or a different cache manager in a system differentfrom a system containing cache manager 104. For example, cache manager104 is included in a first system that includes L1 cache 108 and asecond system that includes another cache manager and another L1 cachethat are a part of the caching hierarchically associated with the sameL2 cache 112 and storage 114. Components not shown in FIG. 1 may alsoexist.

FIG. 2 is a flow chart illustrating an embodiment of a process forprocessing a request for a data record. In some embodiments, the processof FIG. 2 is implemented in cache manager 104 of FIG. 1.

At 202, a request for a data record is received. In some embodiments,the request is received from a requestor such as requestor 102 ofFIG. 1. The requested data record may be any type of data including adatabase record, a key-value pair, a file, a text, a value, an image,and any other type of data.

At 204, it is determined whether the requested data record is stored inan L1 cache. In some embodiments, the L1 cache is included in L1 cache108 of FIG. 1. If at 204 it is determined that the requested data recordis stored in the L1 cache, at 206, the requested data record stored inthe L1 cache is obtained from the L1 cache and returned to therequester. If at 204 it is determined that the requested data record isnot stored in the L1 cache, at 208, it is determined whether therequested data record is stored in an L2 cache. In some embodiments, theL2 cache is included in L2 cache 112 of FIG. 1.

If at 208 it is determined that the requested data record is stored inthe L2 cache, at 210, the requested data record stored in the L2 cacheis obtained from the L2 cache and returned to the requester.

At 212, it is determined whether to cache/store the requested datarecord in the L1 cache and the requested data record is cached in the L1cache, if applicable. In some embodiments, determining whether to cachethe data record includes determining a random value used to make aprobabilistic determination on whether to store the requested datarecord in the L1 cache. In some embodiments, caching the data record inthe L1 cache includes evicting/removing/replacing a previously storeddata record in the L1 cache. In various embodiments, the previouslystored data record is selected based on a time/order indicator (e.g.,associated with when the data record has been last accessed and/orplaced in the cache), a least recently used (LRU) order,last-in-first-out order (LIFO), first-in-first-out order (FIFO), and/ora random order.

If at 208 it is determined that the requested data record is not storedin the L2 cache, at 214, the requested data record is obtained from asource storage and returned to the requester. In some embodiments, thesource storage is included in storage 114 of FIG. 1. In otherembodiments, one or more other levels of cache or other cache may existand these cache(s) are checked before the requested data is obtainedfrom the source storage.

At 216, it is determined whether to cache/store the requested datarecord in the L2 cache and the requested data record is cached in the L2cache, if applicable. In some embodiments, determining whether to cachethe data record includes determining a random value used to make aprobabilistic determination on whether to store the requested datarecord in the L2 cache. In some embodiments, caching the data record inthe L2 cache includes evicting/removing/replacing a previously storeddata record in the L2 cache. In various embodiments, the previouslystored data record is selected based on a time indicator (e.g.,associated with when the data record has been last accessed and/orplaced in the cache), least recently used (LRU), last-in-first-out order(LIFO), first-in-first-out order (FIFO), and/or a random order. In someembodiments, at 216, it is also determined whether to cache/store therequested data record in the L1 cache and the requested data record iscached in the L1 cache, if applicable.

FIG. 3 is a flow chart illustrating an embodiment of a process fordetermining whether to cache a data record in a cache storage. In someembodiments, the process of FIG. 3 is implemented in cache manager 104of FIG. 1. In some embodiments, the process of FIG. 3 is included instep 212 and/or 216 of FIG. 2.

At 302, a random value associated with a potential data record to bestored in a cache is determined. In some embodiments, the cache includesL1 cache 108 of FIG. 1. In some embodiments, the cache includes L2 cache112 of FIG. 1. In some embodiments, determining the random valueincludes determining a random value bound by a predetermined thresholdvalue. For example, a random integer value between 1 and a thresholdvalue is determined. In some embodiments, the random value is associatedwith a statistical probability the data record should be stored in thecache each time it is determined that the data record has been requestedbut not stored in the cache. The threshold value may be configured toadjust the statistical probability that the data record will be storedin the cache. In some embodiments, the threshold value is predetermined.In some embodiments, the threshold value is dynamically determined. Insome embodiments, the threshold value is determined based at least inpart on an analysis of a historical access pattern of one or more datarecords in a data set. For example, number of accesses for each datarecord in the data set is determined and used to determine the thresholdvalue. In some embodiments, the random value and/or threshold value isbased at least in part on a storage size and/or hierarchy level of thecache. In some embodiments, the random value must be an integer value.In some embodiments, the threshold value is based at least in part on aprobabilistic function to be used for a particular cache. For example,each cache level of a plurality of cache levels is associated with adifferent probabilistic function used to determine whether to cache thedata record in the particular cache level, and the probabilisticfunction for each cache level is associated with a different thresholdvalue.

At 304, a comparison value based on the random value is compared with areference value. In some embodiments, the comparison value is the randomvalue. In some embodiments, the comparison value is computed using therandom value. For example, the comparison value is the random valuemodulo the threshold value. In some embodiments, the reference value isa predetermined value that represents the value of the comparison valueif the data record should be stored in the cache. In some embodiments,performing the comparison includes determining whether the comparisonvalue equals the reference value. Other comparisons such as greaterthan, less than, greater than or equal to, or less than or equal tocomparisons may be performed. In some embodiments, the statisticalprobability that the comparison value is of a certain comparison (e.g.,equal to) as compared to the reference value is associated with theprobability that the data record of interest should be stored in thecache. For example, the random value must be an integer value between 1and a threshold value, and if the comparison value (e.g., the randomvalue modulo the threshold value) equals the reference value (e.g.,integer value of 1), the data record of interest is to be stored in thecache. Otherwise, the data record of interest is not yet to be stored inthe cache.

At 306, if it is determined that a result of the comparison matches acriteria, at 308, it is determined to store the data record of interestin the cache. In some embodiments, matching the criteria includesdetermining whether in step 304 the comparison value matches a referencevalue. In some embodiments, storing the data record in the cacheincludes replacing and/or removing an existing data record stored in thecache. The existing data record to be replaced/removed may be selectedbased on a time indicator (e.g., associated with when the data recordhas been last accessed and/or placed in the cache), least recently used(LRU), last-in-first-out order (LIFO), first-in-first-out order (FIFO),and/or a random order.

At 306, if it is determined that a result of the comparison does notmatch a criteria, at 310, it is determined not to store the data recordof interest in the cache. In some embodiments, the criteria does notmatch because in step 304 the comparison value does not match areference value.

FIG. 4 is a flow chart illustrating an embodiment of a process fordetermining a threshold value. In some embodiments, the process of FIG.4 is implemented in cache manager 104 of FIG. 1. In some embodiments,the process of FIG. 4 is included in step 212 and/or 216 of FIG. 2. Insome embodiments, the process of FIG. 4 is included in step 302 of FIG.3.

At 402, a data pattern of data records in a data set is analyzed. Insome embodiments, analyzing the data pattern includes obtaining/trackinga data access pattern. For example, for a predetermined period of time,the number of times each record in a data set is accessed is tracked. Insome embodiments, by tracking the data pattern of a data set for alimited period of time, an appropriate threshold value can be determinedfor the data set and a data access pattern. In some embodiments,analyzing the data set includes organizing, sorting, and/or processingthe data pattern. In some embodiments, the total number of unique datarecords that were accessed a given number of times or less isdetermined. For example, it is determined that 90% of unique datarecords are accessed for no more than 32 times per day. In someembodiments, the total number of data record accesses for data record(s)that were accessed a given number of times or less is determined. Forexample, it is determined that 79% of accesses are from the data recordswhich are accessed for more than 32 times per day. In some embodiments,performing analysis includes generating a graph or other visualizationof the data record access pattern that can be used to determine anaccess pattern value.

At 404, an access pattern value is determined based on the analysis. Insome embodiments, determining the access pattern value includesdetermining a value that represents a desired average number of timesthe data record has been accessed and not stored in the cache during agiven amount of time before the data record is stored in a given levelof cache. For example, 32 is selected as the access pattern valuebecause the analysis performed in step 402 determined that 90% of uniquedata records are accessed for no more than 32 times per day and that 79%of accesses are from the data records which are accessed for more than32 times per day. In some embodiments, a plurality of access patternvalues may be determined. In some embodiments, determining the accesspattern value includes determining a value that represents a number ofaccesses during a given amount of time associated with a data recordthat should be stored in a given level of cache with a relatively lowdegree of probability. In some embodiments, determining the accesspattern value includes determining a value that represents a number ofaccesses during a given amount of time associated with a data recordthat should be stored in a given level of cache with a relatively higherdegree of probability. For example, it is determined that when thenumber of accesses of a given data record is 10, the probability thedata record is stored in a certain cache should be relatively low (e.g.,less than 30%), when the number of accesses of the given data record is32, the probability the data record is stored in a certain cache shouldbe relatively high (e.g., around 60%), and when the number of accessesof the given data record is 100, the probability the data record isstored in a certain cache should be relatively very high (e.g., greaterthan 90%). In various embodiments, the access pattern value isdetermined using various probability density functions. For example, theaccess pattern value is determined using a binomial density functionand/or another density function.

At 406, using the access pattern value, a threshold value is determined.In some embodiments, the determined threshold value is the thresholdvalue used in step 212 and/or 216 of FIG. 2 and/or step 302 of FIG. 3.For example, for each access of a data record that is not stored in agiven cache, the data record is stored in the given cache if a randomlygenerated integer between 1 and the threshold value (i.e., N) modulo Nequals to 1. This gives the probability of 1/N a data record is storedin the given cache and (N−1)/N probability the data record is not storedin the cache. As a data record is accessed more often, it has a greaterchance to be promoted because the probabilistic determination of whetherto store the data record in the cache is performed each time the datarecord is accessed but not stored in the cache. The calculation belowshows the probability a data record will stored in the cache.

Given:

P(S=k) is the probability a data record is stored in a given cache whenit gets accessed at k_(th) time.P(F=k) is the probability a data record is not stored in a given cachewhen it gets accessed at k_(th) time.

The probability for each individual trial:

${P_{s} = \frac{1}{N}},{P_{f} = \frac{N - 1}{N}}$

where P_(s) is the success probability and P_(f) is the failureprobability

${P\left( {S = k} \right)} = {{{P\left( {F = {k - 1}} \right)}*P_{s}} = {{\frac{1}{N}*P_{f}*{P\left( {F = {k - 2}} \right)}} = {{\frac{1}{N}*\left( P_{f} \right)^{k - 1}} = {\frac{1}{N}*\left( \frac{N - 1}{N} \right)^{k - 1}}}}}$

The cumulative probability:

$\quad\begin{matrix}{{P\left( {S \leq k} \right)} = {{P\left( {S = 1} \right)} + {P\left( {S = 2} \right)} + \ldots + {P\left( {S = k} \right)}}} \\{= {\frac{1}{N} + {\frac{1}{N}*\frac{N - 1}{N}} + {\frac{1}{N}*\left( \frac{N - 1}{N} \right)^{2}} + \ldots + {\frac{1}{N}*\left( \frac{N - 1}{N} \right)^{k - 1}}}} \\{= \frac{\frac{1}{N}*\left( {1 - \left( \frac{N - 1}{N} \right)^{k - 1}} \right)}{1 - \left( \frac{N - 1}{N} \right)}} \\{= {1 - \left( \frac{N - 1}{N} \right)^{k - 1}}}\end{matrix}$

For example, the access pattern values of 10, 32, and 100 have beendetermined because it is determined that when the number of accesses ofa given data record is 10, the probability the data record is stored ina certain cache should be relatively low (e.g., less than 30%), when thenumber of accesses of the given data record is 32, the probability thedata record is stored in a certain cache should be relatively high(e.g., around 60%), and when the number of accesses of the given datarecord is 100, the probability the data record is stored in a certaincache should be relatively very high (e.g., greater than 90%). Usingthese access pattern values and associated criteria, a threshold valueof 32 (i.e., N=32) has been determined. Using the earlier calculatedprobability formula:

${P\left( {S \leq 10} \right)} = {{1 - \left( \frac{32 - 1}{32} \right)^{10 - 1}} = {24.85\%}}$${P\left( {S \leq 32} \right)} = {{1 - \left( \frac{32 - 1}{32} \right)^{32 - 1}} = {62.63\%}}$${P\left( {S \leq 100} \right)} = {{1 - \left( \frac{32 - 1}{32} \right)^{100 - 1}} = {95.69\%}}$

This means that a data record has 24.85% probability to be stored in acache with 10 accesses, 62.63% probability to be stored in the cachewith 32 accesses and 95.69% probability to be stored in the cache with100 accesses. These values are merely an example. Other access patternvalues and associated desired cache storage probability may be used. Insome embodiments, the threshold value may be determined at least in partbased on a size and/or hierarchy level of the cache. For example, alower probability of being stored in the cache is associated with asmaller and/or higher level cache (e.g., L1 cache) as compared to alarger and/or lower level cache (e.g., L2 cache).

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

What is claimed is:
 1. A system for storing data in a cache, comprising:a processor configured to: determine that a data record is not stored ina cache; generate a random value using a threshold value; and determinewhether to store the data record in the cache based at least in part onthe generated random value; and a memory coupled to the processor andconfigured to provide the processor with instructions.
 2. The system ofclaim 1, wherein a probability that the random value is associated witha certain reference value approximates a desired probability that thedata record will be stored in the cache.
 3. The system of claim 1,wherein there exists a chance of$\frac{1}{{the}\mspace{14mu} {threshold}\mspace{14mu} {value}}$ thedata record will be stored in the cache when the determination is madeon whether to store the data record in the cache based at least in parton the generated random value.
 4. The system of claim 1, wherein thethreshold value has been preconfigured.
 5. The system of claim 1,wherein the threshold value has been determined dynamically.
 6. Thesystem of claim 1, wherein the threshold value has been determined basedat least in part on an observed data access pattern of a data set thatincludes the data record.
 7. The system of claim 1, wherein thethreshold value has been determined based at least in part on a datapattern value that represents a desired average number of times the datarecord has been accessed and not stored in the cache before the datarecord is stored in the cache.
 8. The system of claim 1, wherein thethreshold value has been determined based at least in part on a size ofthe cache.
 9. The system of claim 1, wherein the threshold value hasbeen determined based at least in part on a hierarchy level of thecache.
 10. The system of claim 1, wherein in the event it is determinedto store the data record in the cache, the processor is furtherconfigured to select an existing data record stored in the cache to bereplaced with the data record to be stored in the cache.
 11. The systemof claim 10, wherein the existing data record is selected based on anindicator that is associated with when the existing data record was lastaccessed from the cache.
 12. The system of claim 1, wherein generatingthe random value is associated with a probabilistic determination onwhether to store the data record in the cache.
 13. The system of claim1, wherein generating the random value using the threshold valueincludes generating a random integer value between 1 and the thresholdvalue.
 14. The system of claim 1, wherein determining whether to storethe data record in the cache includes comparing the random value with areference value.
 15. The system of claim 1, wherein determining whetherto store the data record in the cache includes comparing a referencevalue with a result of the random value modulo the threshold value. 16.The system of claim 15, wherein it is determined to store the datarecord in the cache if the result matches the reference value.
 17. Thesystem of claim 1, wherein the processor is further configured to:generate a second random value using the threshold value; and determinewhether to store the data record in the cache based at least in part onthe generated second random value.
 18. The system of claim 1, whereinthe processor is further configured to: determine that the data recordis not stored in a second cache; generate a second random value using asecond threshold value; and determine whether to store the data recordin the second cache based at least in part on the generated secondrandom value.
 19. A method of storing data in a cache, comprising:determining that a data record is not stored in a cache; generating arandom value using a threshold value; and determining whether to storethe data record in the cache based at least in part on the generatedrandom value.
 20. A computer program product for storing data in acache, the computer program product being embodied in a tangiblecomputer readable storage medium and comprising computer instructionsfor: determining that a data record is not stored in a cache; generatinga random value using a threshold value; and determining whether to storethe data record in the cache based at least in part on the generatedrandom value.